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LINEAR  DECISION  AND  CONTROL^ 


By  C.  Duncan  MacRae 


The  linear  decision  problem[3]  is  to  choose  at  the  end  of  period 
k-1  a  vector  of  instrument,  or  control,  variables,  u^.  -i »  given 
observations  on  u,  »,  u,  _, . . .  and  on  y.  ,,  yif_2»"*»  ^°^   k«l,2,...,N, 
where  u,  ^  is  a  vector  of  partly  controlled,  or  endogenous,  variables, 
so  as  to  minimize  the  expected  value  of  J,  where 

(1)         J  =  t'u  +  s'y  +  J5[u'Ru  +  y'Qy  +  u'Py  +  y'P'u] 
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subject  to 


(2)  y  =  Hu  +  h. 


or 


k-1 


^k  "     .lo'^.j+l^j  ■"  Vl      k=1.2.....N. 


where  s,  t,  R,  Q,  P,  and  H  are  given,  R  and  Q  are  symmetric,  and  h 
is  random  with  given  mean  and  variance.  The  purpose  of  this  note  is 
to  transform  the  linear  decision  problem  into  a  stochastic  linear 
control  problem  and,  then,  to  derive  the  optimal  decision  rules  with 
dynamic  programming.  The  advantages  of  transformation  are  in  problem 
formulation,  solution,  analysis,  and  generalization. 

Consider  the  reduced  form  representation  of  a  stochastic  dy- 
namic linear  system: 

n 

^k  -  ^^l^  ^j.k-i^k-j  -^  ^j.k-iVj  -^  ^j,k-iVjl 

"*■  °l,k-lVl  k-1, 2,...   , 


where  v    is  a  vector  of  exogenous  variables,  e    is  a  vector  of 
random  variables  with  mean  zero  and  variance  ^i,  i  »  ^-  v  i  •  ^4  l-  i » 
C  j^_j^,  D^  ^_^,   \-i»  ^"'^  '^k-i  "®  given  for  j»l,2,...,n  and  k-1, 2,..., 
and  the  predetermined  variables  as  of  period  k  are  associated  with 
period  k-j  for  j=l,2,.,,,n.  A  state  space  representation  of  the 
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system  is 


(3) 


Vi  "  Vk"' Vk-"  Vk^  Vk       k==o,i,...    , 


where  x,    is  the  state  of  the  system. 


\     "      f^k'^k-] 
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^l,k  ^2,k  •••  S-l,kS,k 


l.k 


X   =  inax{n,N},  A  ,  ,  B  ,  ,  and  C  ,  are  zero  for  j  >  n,  and  x-  Is  given. 

The  final  form  representation  of  the  system[3,4]  then  is  given  by  (2),  where 


^k  '  Vic'      \.j+i   =  \\,i^i^y 


and 


k.j      ,:j  i 


k-l 
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H  A^  ,    *, 
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The  linear  decision  problem  transformed  into  a  linear  control 


problem,  therefore,  is  to  choose  u   ,  given  x  ,  for  k=l,2 N  so 

as  to  minimize  the  expected  value  of  J  subject  to  (3),  where 
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Q  is  symmetric  positive  semi-definite,  R,  is  symmetric  positive 
definite,  and  Q,  ,  ,,  and  P,  ,  ,  are  zero  for  k  and  k'  <  1. 
Applying  dynamic  programming [1]  let 


(^)  -     ^ME{x^IVi>IIq     +     S-AVl  ^  ^^^N>N-1>^N  -^  '^J-lS 

N 

+     J5var{x^Q^  +  x^Sj^|x^_^} 
<5)  ■=     ^N-l^-l^A-l^-l  •"  *^N-lf«N-lVN-l  ^  V\-l 

+  ^-if^i-i^N  ^  ViVn-iVi^ 

■^  Wl-lfVl^A-l^-l  -^  ^N-AS-l^N-1  -^  ^N-l^N  -^  '^N^ 
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■^  ^^N-l^N-l\^N-l^N-l  ■*■  ^N-l^N-l^N 


+  ^»^'<"n-i°n-iVn-i>  • 


Minimizing  y     with  respect  to  u^  ^  given  x  -  we  obtain 


■^«n-iVn-iVi-^«n-i^n  + V' 


where  u    is  the  optimal  u^  - .  Then  substituting  (6)  into  (5)  and 
rearranging  terms  we  get 

^  ^^tVl^N  ^  S^ '  f  ViVn-1  -^  \]"'f^N-l^N  +  'n^ 
+  ^tr{«^_^D'_^Q^D^_^}  . 


where 
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and 


^«>  \-i  '  Vi  ^  ^-i^k  ^  N^-iVk-i\-i 


.-1 


for  k=N  with  K.  =Q„  and  g  =s„,  and  y»,  is  the  optimal  value  of  y„.     Then 

N   N       M   N        N  N 

(9)         Yj^_,  .  E{\.i(VrV2^  -^  \l V2> 

+  var{x^_^K^_^Xj^_^  +  ^-iSk-l'Va^  "^  ^ 
for  k=N,  where 


(10)   «,  -  J^('.[Cj.i^j.il'[Kj-KjBj.llBj.,KjBj.,*Rjl-^Bj.,KjHC^.lZj.ll 

*  [Cj.jZj.,l'[gj-KjB^.l[B-.,KjBj.,  +Kjr^[B^.l8j  *  t^l] 

*  MB,.,gj  *  tjl'tBj.iQjB^.l  *  ■'jl-'CB^.iSj  +  tjl 


and  with  the  exception  of  c,  is  of  the  game  form  as  (4) . 
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Hence, 


(11)  u^_^     .     -   [B'_,K^Bj^_3^  +  \l"'[»k-AVlVl 


+  «i-iVk-i Vi  -^  ^i-i^k  -^  \^  ^"«'«-^'  •  •  •  '^  ' 


(12)  Y^_^     =     K-2fV2  -  \-2^V2  -^  ^k-2f\-2   ~  ^^-1^   "^  \-l 

k=N+l,N,...,2    , 


where  K   ,  g,  ,,  Y.  ,.  and  «  are  determined  by  (7),  (8),  (9),  and  (10), 
respectively,  the  Ricattl  matrix,  IL   ,is  symmetric  positive  semi-definite, 

■k 

Y,  ,  is  the  optimal  value  of  y   ,  and  the  optimal  evolution  of  the  state 
of  the  system  is  described  by 


Ci  =   t^  -  \£^i\+i\  ^  \+i3"\\+iHVk  ^  Vk^ 
-  \f^iVi\  ^  \+i'''f^k^k+i  ^  Vi^ 


+  Dj^Ej^  k=0,l,...,N-l 


The  structure  of  the  optimal  strategy  may  be  seen  immediately 
from  (11).  Certainty  equivalence  holds;  hence,  D    and  Q.         play  no  r41e 
in  the  strategy,  although  the  expected  gain  from  minimizing  expected  util- 
ity[3],  or  the  expected  value  of  perfect  information  on  e,  .[2],  is 
i5tr{f2   D'  -.Q^D,  ,}.  There  is  feedback  on  x    and  z,_,f  which  are  direct- 
ly observable,  rather  than  on  past  errors[3,5].  There  is  feedforward  on 
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future  0  ,  R  ,  A^,  and  B  through  K  and  on  future  s  ,  t  ,  C  ,  and  z, 
through  g,  .   Computation  of  the  strategy  is  done  recursively  through  (7) 
and  (8)  and  involves  inversion  of  matrices  only  of  order  equal  to  the 


number  of  control  variables.  From  (12)  we  see  that  the  optimal  expected 

*  * 

and  the  imputed  price  of  x 


*  * 

Value  of  J  is  y  and  the  imputed  price  of  x  for  k=N,N-l, . . . ,1  is 


The  problem  may  be  generalized  to  include  objective  functions  of  the 
form 

(13)     J  =   t'[u-u]  +  s'[y-y]  +  J5[[u-u]'R[u-u]  +  [y-y]'Q[y-y] 
+  [u-fl]'P[y-y]  +  [y-y]'P'[u-fi]]   , 

where  u  and  y  are  the  desired  levels  of  u  and  y,  respectively,  since  (13) 
is  of  the  same  form  as  (1)  with 

t  =  £  -  Ru  -  Py,     s  =  s  -  Oy  -  P'u 

and  the  addition  of  a  constant  term.  Moreover,  since  the  state-space 
representation,  in  contrast  to  the  final  form  representation,  corresponds 
directly  to  the  reduced  form  and  the  state  of  the  system  is  observable 
the  problem  may  also  be  generalized  to  include  unknown  parameters  or 
errors  in  variablesfl] . 
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